<!-- HTML header for doxygen 1.8.6-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<title>OpenCV: How to schedule your network for Halide backend</title>
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../jquery.js"></script>
<script type="text/javascript" src="../../dynsections.js"></script>
<script type="text/javascript" src="../../tutorial-utils.js"></script>
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../search/searchdata.js"></script>
<script type="text/javascript" src="../../search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
//<![CDATA[
MathJax.Hub.Config(
{
  TeX: {
      Macros: {
          matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
          fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
          forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
          forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
          vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
          vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
          cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
          distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
          distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
          hdotsfor: ["\\dots", 1],
          mathbbm: ["\\mathbb{#1}", 1],
          bordermatrix: ["\\matrix{#1}", 1]
      }
  }
}
);
//]]>
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<!--#include virtual="/google-search.html"-->
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
  <td style="padding-left: 0.5em;">
   <div id="projectname">OpenCV
   &#160;<span id="projectnumber">4.5.2</span>
   </div>
   <div id="projectbrief">Open Source Computer Vision</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
</script>
<script type="text/javascript" src="../../menudata.js"></script>
<script type="text/javascript" src="../../menu.js"></script>
<script type="text/javascript">
$(function() {
  initMenu('../../',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="../../d9/df8/tutorial_root.html">OpenCV Tutorials</a></li><li class="navelem"><a class="el" href="../../d2/d58/tutorial_table_of_content_dnn.html">Deep Neural Networks (dnn module)</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">How to schedule your network for Halide backend </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p><b>Prev Tutorial:</b> <a class="el" href="../../de/d37/tutorial_dnn_halide.html">How to enable Halide backend for improve efficiency</a></p>
<p><b>Next Tutorial:</b> <a class="el" href="../../d0/d6c/tutorial_dnn_android.html">How to run deep networks on Android device</a></p>
<table class="doxtable">
<tr>
<th align="right"></th><th align="left"></th></tr>
<tr>
<td align="right">Original author </td><td align="left">Dmitry Kurtaev </td></tr>
<tr>
<td align="right">Compatibility </td><td align="left">OpenCV &gt;= 3.3 </td></tr>
</table>
<h2>Introduction</h2>
<p>Halide code is the same for every device we use. But for achieving the satisfied efficiency we should schedule computations properly. In this tutorial we describe the ways to schedule your networks using Halide backend in OpenCV deep learning module.</p>
<p>For better understanding of Halide scheduling you might want to read tutorials @ <a href="http://halide-lang.org/tutorials">http://halide-lang.org/tutorials</a>.</p>
<p>If it's your first meeting with Halide in OpenCV, we recommend to start from <a class="el" href="../../de/d37/tutorial_dnn_halide.html">How to enable Halide backend for improve efficiency</a>.</p>
<h2>Configuration files</h2>
<p>You can schedule computations of Halide pipeline by writing textual configuration files. It means that you can easily vectorize, parallelize and manage loops order of layers computation. Pass path to file with scheduling directives for specific device into <code><a class="el" href="../../db/d30/classcv_1_1dnn_1_1Net.html#a56fbff351d1e0a47fb5aabf6915fc279" title="Compile Halide layers. ">cv::dnn::Net::setHalideScheduler</a></code> before the first <code><a class="el" href="../../db/d30/classcv_1_1dnn_1_1Net.html#a98ed94cb6ef7063d3697259566da310b" title="Runs forward pass to compute output of layer with name outputName. ">cv::dnn::Net::forward</a></code> call.</p>
<p>Scheduling configuration files represented as YAML files where each node is a scheduled function or a scheduling directive. </p><div class="fragment"><div class="line">relu1:</div><div class="line">  reorder: [x, c, y]</div><div class="line">  split: { y: 2, c: 8 }</div><div class="line">  parallel: [yo, co]</div><div class="line">  unroll: yi</div><div class="line">  vectorize: { x: 4 }</div><div class="line">conv1_constant_exterior:</div><div class="line">  compute_at: { relu1: yi }</div></div><!-- fragment --><p>Considered use variables <code>n</code> for batch dimension, <code>c</code> for channels, <code>y</code> for rows and <code>x</code> for columns. For variables after split are used names with the same prefix but <code>o</code> and <code>i</code> suffixes for outer and inner variables correspondingly. In example, for variable <code>x</code> in range <code>[0, 10)</code> directive <code>split: { x: 2 }</code> gives new ones <code>xo</code> in range <code>[0, 5)</code> and <code>xi</code> in range <code>[0, 2)</code>. Variable name <code>x</code> is no longer available in the same scheduling node.</p>
<p>You can find scheduling examples at <a href="https://github.com/opencv/opencv_extra/tree/master/testdata/dnn">opencv_extra/testdata/dnn</a> and use it for schedule your networks.</p>
<h2>Layers fusing</h2>
<p>Thanks to layers fusing we can schedule only the top layers of fused sets. Because for every output value we use the fused formula. In example, if you have three layers Convolution + Scale + ReLU one by one, </p><div class="fragment"><div class="line">conv(x, y, c, n) = sum(...) + bias(c);</div><div class="line">scale(x, y, c, n) = conv(x, y, c, n) * weights(c);</div><div class="line">relu(x, y, c, n) = max(scale(x, y, c, n), 0);</div></div><!-- fragment --><p>fused function is something like </p><div class="fragment"><div class="line">relu(x, y, c, n) = max((sum(...) + bias(c)) * weights(c), 0);</div></div><!-- fragment --><p>So only function called <code>relu</code> require scheduling.</p>
<h2>Scheduling patterns</h2>
<p>Sometimes networks built using blocked structure that means some layer are identical or quite similar. If you want to apply the same scheduling for different layers accurate to tiling or vectorization factors, define scheduling patterns in section <code>patterns</code> at the beginning of scheduling file. Also, your patters may use some parametric variables. </p><div class="fragment"><div class="line"># At the beginning of the file</div><div class="line">patterns:</div><div class="line">  fully_connected:</div><div class="line">    split: { c: c_split }</div><div class="line">    fuse: { src: [x, y, co], dst: block }</div><div class="line">    parallel: block</div><div class="line">    vectorize: { ci: c_split }</div><div class="line"># Somewhere below</div><div class="line">fc8:</div><div class="line">  pattern: fully_connected</div><div class="line">  params: { c_split: 8 }</div></div><!-- fragment --><h2>Automatic scheduling</h2>
<p>You can let DNN to schedule layers automatically. Just skip call of <code><a class="el" href="../../db/d30/classcv_1_1dnn_1_1Net.html#a56fbff351d1e0a47fb5aabf6915fc279" title="Compile Halide layers. ">cv::dnn::Net::setHalideScheduler</a></code>. Sometimes it might be even more efficient than manual scheduling. But if specific layers require be scheduled manually, you would be able to mix both manual and automatic scheduling ways. Write scheduling file and skip layers that you want to be scheduled automatically. </p>
</div></div><!-- contents -->
<!-- HTML footer for doxygen 1.8.6-->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Apr 2 2021 11:36:34 for OpenCV by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
</a> 1.8.13
</small></address>
<script type="text/javascript">
//<![CDATA[
addTutorialsButtons();
//]]>
</script>
</body>
</html>
